This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space. As a result, after training, our model can generate different realistic versions of an input image by varying the attribute values. By using continuous attribute values, we can choose how much a specific attribute is perceivable in the generated image. This property could allow for applications where users can modify an image using sliding knobs, like faders on a mixing console, to change the facial expression of a portrait, or to update the color of some objects. Compared to the state-of-the-art which mostly relies on training adversarial networks in pixel space by altering attribute values at train time, our approach results in much simpler training schemes and nicely scales to multiple attributes. We present evidence that our model can significantly change the perceived value of the attributes while preserving the naturalness of images.

This paper addresses the problem of manipulating images using natural language description. Our task aims to semantically modify visual attributes of an object in an image according to the text describing the new visual appearance. Although existing methods synthesize images having new attributes, they do not fully preserve text-irrelevant contents of the original image. In this paper, we propose the text-adaptive generative adversarial network (TAGAN) to generate semantically manipulated images while preserving text-irrelevant contents. The key to our method is the text-adaptive discriminator that creates word level local discriminators according to input text to classify fine-grained attributes independently. With this discriminator, the generator learns to generate images where only regions that correspond to the given text is modified. Experimental results show that our method outperforms existing methods on CUB and Oxford-102 datasets, and our results were mostly preferred on a user study. Extensive analysis shows that our method is able to effectively disentangle visual attributes and produce pleasing outputs.

After rebuttal comments: * readability: I trust the authors to update the paper based on my suggestions (as they agreed to in their rebuttal). For AttrGAN, they did change the weight sweep and for SISGAN they used the same hyperparameters as they used in their method (which I would object to in general, but given that the authors took most of their hyperparameters from DCGAN, does not create an unfair advantage). I expect the additional details of the experimental results to be added in the paper (as supplementary material). Ensure that content that is not relevant to the text does not change. Method: to avoid changing too much of the image, use local discriminators that learn the presence of individual visual attributes.

These attributes are known at training time and for a dataset of faces include aspects like [old vs young], [smiling vs not smiling], etc. They hope to be able to tweak these attributes along a continuous spectrum, even when the labels only occur as binary values. To achieve this they propose an (encoder, decoder) setup where the encoder maps the image x to a latent vector z and then the decoder produces an image taking z, together with the attributes y as inputs. When such a network is trained in the ordinary fashion, the decoder learns to ignore y because z already encodes everything that the network needs to know. To compel the decoder network to use y, the authors propose introducing a adversarial learning framework in which a discriminator D is trained to infer the attributes from z. Thus the encoder must produce representations that are invariant to the attributes y. The writing is clear and any strong researcher should be able to reproduce their results from the presentation here.

This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space. As a result, after training, our model can generate different realistic versions of an input image by varying the attribute values. By using continuous attribute values, we can choose how much a specific attribute is perceivable in the generated image. This property could allow for applications where users can modify an image using sliding knobs, like faders on a mixing console, to change the facial expression of a portrait, or to update the color of some objects. Compared to the state-of-the-art which mostly relies on training adversarial networks in pixel space by altering attribute values at train time, our approach results in much simpler training schemes and nicely scales to multiple attributes.

This paper addresses the problem of manipulating images using natural language description. Our task aims to semantically modify visual attributes of an object in an image according to the text describing the new visual appearance. Although existing methods synthesize images having new attributes, they do not fully preserve text-irrelevant contents of the original image. In this paper, we propose the text-adaptive generative adversarial network (TAGAN) to generate semantically manipulated images while preserving text-irrelevant contents. The key to our method is the text-adaptive discriminator that creates word level local discriminators according to input text to classify fine-grained attributes independently.